C Homework Set 9 Spring 2011 Solutions S S. = S, E ˆβ. = β 1, and Var( ˆβ 1. 1 SOLUTION: Cov(Y, Y i ) = Cov Yj = σ. In the

Transkript

1 . I the smple lear regresso model, we use Y β + β x + ε for,,,. The ε s are assumed to be a sample from a ormal populato wth mea ad stadard devato. The values x, x,, x are assumed o-radom. ce we treat x, x,, x as oradom, the quatty s also oradom. The ftted le wll be deoted Y ˆβ + ˆβ x or perhaps as Y b + b x. Not all sources use the same otato. Assume that you already kow ˆβ, E ˆβ β, ad Var( ˆβ ). (a) how that Cov( Y, Y ). OLUTION: Cov(Y, Y ) Cov Yj, Y Cov (, ) j Yj Y. I the j last summato, oly a sgle summad, where j, s dfferet from zero. (b) how that Cov(Y, ˆβ ). HINT: For ay two sets of values u, u,, u ad v, v,, v, each of u u v v u u v. Ths s very easy legth, t happes that ( )( ) ( ) to prove. Also, ( u u)( v v) u ( v v) OLUTION: ˆβ ( x x)( Y Y) ( x ) x Y cy x x I ths form, c ad we have wrtte ˆβ as a lear fucto of the depedet radom varables Y, Y,, Y. gs

2 The, usg part (a), Cov(Y, ˆβ ) Cov(Y, cy ) ( x x ) c c x x (c) how that Cov( ˆβ, ˆβ ) x. OLUTION: Cov( ˆβ, ˆβ ) Cov(Y - ˆβ x, ˆβ ) Cov(Y, ˆβ ) - x Cov( ˆβ, ˆβ ) Cov(Y, ˆβ ) - x Var( ˆβ ) Cov(Y, ˆβ ) x x. As the prevous problem, we use the smple lear regresso model Y β + β x + ε for,,,. The ε s are assumed to be a sample from a ormal populato wth mea ad stadard devato. The values x, x,, x are assumed o-radom. (a) how that E (M ) + β. HINT: Recall that, wth oe predctor, M wll help to show frst that ( x x) OLUTION: Frst, we ll verfy the ht. β + ε. ˆ ( β ). It ( x x) Y ( x x)( β +β x +ε) ( x x)( Y Y) β + β + ε β + ( x x) ε ( x x) ( x x) x ( x x) We ote that β s oradom, whle ( x x ) zero. ε s radom wth expected value gs

3 ( ce M ) E ( ) Var ( ) ( E( )) The E., t follows that we eed to fd ( ) +. From the ht, t s clear that E( ) β. Var( ) Var ( x x)( Y Y ) Var ( x ) ( x x) Var ( Y) ( x ) x x Y E ( ) Thus ( ) E ( ) + β ( ) + β ( ) + β ad fally E M + β. (b) how that E(M Resd ). HINT: Fd E ( ). The get E( Resd ) by subtracto. OLUTION: E ( ) E ( ) Y Y E β +β x +ε β +β x + ε E β ( x x) + ( ε ε) ( ) ( ) ( ) { } E β ( x x) + β ( x x)( ε ε ) + ( ε ε) β ( x x) + β E ( x x)( ε ε ) + E ( ε ε) The frst term s just β. The secod term has expected value zero from ε ε. The thrd term s just a sum of squares aroud a average; t s the kd of calculato used to estmate. 3 gs

4 ( ) β + ce + Resd, we have foud that E Resd E - E R ( ) β + - { + β } ( - ) Thus, E M Resd E Res. 3. Use the same smple lear regresso model as the frst two problems. Be sure to dstgush carefully betwee the ose term ε ad the resdual e Y Y ˆ. Fd the dstrbuto of e. Note also E e ad Var e. You may assume that Cov( ˆβ, ε ) ad also Cov( ˆβ, ε ) ; these eed a careful proof, but you ca assume them here. OLUTION: Let s defe the th resdual as e Y Y ˆ. We are asked for the varace of e. Whle we re at t, we may as well ote the etre dstrbuto. ce Y ˆ s a lear combato of Y, Y,, Y, t happes that e s also lear Y, Y,, Y. Thus e s ormally dstrbuted uder the usual model. All that remas s fdg the mea ad varace of e. Note that Y ˆ ˆβ + ˆβ x, so that e Y Y ˆ β + β x + ε - ( ˆβ + ˆβ x ) (β - ˆβ ) + (β - ˆβ ) x + ε. ce E ˆβ β, E ˆβ β, ad E ε, we ca verfy that E e. Now for the varace: Var e Var [ (β - ˆβ ) + (β - ˆβ ) x + ε ] Var [ ( - ˆβ ) + ( - ˆβ ) x + ε ] Var [ ( - ˆβ ) ] + Var [ ( - ˆβ ) x ] + Var [ ε ] + Cov( - ˆβ, - ˆβ x ) + Cov( - ˆβ, ε ) + Cov( - ˆβ x, ε ) 4 gs

5 Var [ ˆβ ] + x Var [ ˆβ ] + Var [ ε ] + x Cov( ˆβ, ˆβ ) + + x x + x x x x x x + + ( x x x + + x ) + + ( x x) Thus we ca coclude that e ~ N, + + ( x x). The resduals have (relatvely) smaller varace for those x pots ear x. 4. I a geeral multple lear regresso, establsh the algebrac relatoshp betwee the F statstc ad the R statstc. You ca use the otato of ths aalyss of varace table. ource of varato Degrees of freedom um quares Mea quares F esso K M K Resdual K Resd Resd M Resd K al + Resd F M M Resd The recall that R. Work from the defto of F : 5 gs

6 OLUTION: F M M Resd Resd / K ( ) / K K K Resd K K K K K K K R K R ce F s a oe-to-oe fucto of R, sgfcace tests could be doe terms of R. Ideed, some have proposed tables of cutoff pots for the ull dstrbuto of R. 5. Usg the smple lear regresso model (see Problem ), suppose that you have collected these data: The regresso equato s Y X Predctor Coef E Coef T P Costat X R-q 34.5% R-q(adj) 33.5% Aalyss of Varace ource DF M F P esso Resdual Error al You have just acqured a ew data pot for whch you see x ew, but you have ot yet observed Y ew. (a) Idetfy the value β, β,, b, b, s ε,. OLUTION: Here β, β, ad are ukow parameters. From the prtout, we observe b 49., b.457, s ε 6.367, ad gs

7 (b) Gve a pot predcto for Y ew symbolc terms. (ymbolc terms refers to the tems gve part (a).) Call ths Y ˆ ew. Provde a umerc predcto for x ew 6. OLUTION: The pot predcto s Y ˆ ew b + b x ew. Here ths s Y ˆ ew x ew, ad for the specfc x ew 6, t s (.457 6) (c) The Ŷ ew of (b) s a radom varable. Fd E Ŷ ew ad Var Ŷ ew. OLUTION: E Y ˆ ew E( b + b x ew ) β + β x ew. The (somewhat harder) Var Y ˆ ew Var( b + b x ew ) Var( b ) + x ew Cov(b, b ) + x Var ( b ) ew + x + + xew x xew + ( x x x + x ) ew ew + ew ( x x) Predctos based o x ew values that are far from x have larger varaces! (d) Gve a 95% predcto terval for Y ew. Ths wll certaly be somethg of the form Ŷ ew ± (expresso). Gve ths symbolcally ad the for the specfc case x ew 6. OLUTION: The crtcal tem here s Y ew β + β x ew + ε ew. That s, the ew value Y ew depeds o a ew ose term ε ew. Now observe that Y ew - Y ˆ ew ( β + β x ew + ε ew ) - ( b + b x ew ) ( β b ) + (β b ) x ew + ε ew Certaly ths shows that E( Y ew Y ˆ ew ). Remember that both Y ew ad Y ˆ ew are radom varables. Ths s a lear combato of ormal radom varables, the orgal Y through Y, so Y ew Yˆ ew follows a ormal dstrbuto. Let s ote the varace. 7 gs

8 Var( Y ew - Y ˆ ew ) Var( ( β b ) + (β b ) x ew + ε ew ) Var( ( β b ) + (β b ) x ew ) + Var( ε ew ) Var( ( β b ) + (β b ) x ew ) + Var( b b x ew ) + Var( b + b x ew ) + Var( Y ˆ ew ) + ( x x) ew ew ( x x) It follows the that Y ew + + Yˆ ew ( x x) ew s dstrbuted as N(, ), the stadard ormal. ce s, the regresso estmate of (the mea square error the aalyss of varace table) s depedet ad has the dstrbuto property ( ) s ~ χ, t Yew Yˆ ew must happe that has the t dstrbuto wth degrees ( xew x) s + + of freedom. Thus Yew Yˆ ew P tα/; < < t α/; α ( xew x) s + + We ca ow costruct the predcto terval for Y ew. Wth probablty α, the ot-yet-observed Y ew wll be the terval Yˆ ew ± tα/; s + + ( x x ) ew (e) ce β ad β are parameters, so s β + β x ew. Gve a pot estmate for β + β x ew symbolc form. Call ths Y ew. Gve ths value for the stuato x ew 6. 8 gs

9 OLUTION: We must use Y ew b + b x ew. The pot estmate s also Y ˆ ew. The umerc predcto at x ew 6 s the The umber we use to predct Y ew s exactly the same as the umber we use to predct E Y ew. (f) The Y ew of (e) s a radom varable. Fd E Y ew ad Var Y ew. OLUTION: ce Y ew s computatoally detcal to Y ˆ ew, t must have the same mea ad varace. Thus E Y ew β + β x ew E Y ew + ew ( x x) (g) Gve a 95% cofdece terval for the parameter combato β + β x ew. Do ths symbolcally ad the for x ew 6. OLUTION: The pot estmate s Y ˆ ew. ce Y ˆ ew ~ N β +β xew, + ew ( x x) we ca costruct Yˆ ( x ) ( x x) ew ew s β +β + freedom. Thus, the α terval for β + β x ew s ew as a t radom varable wth degrees of b + b xew ± tα/; s + ( x x ) ew We have most of what we eed here, but we lack x. The umerc form wll have to rema usolved! 9 gs

10 (h) The tervals (d) ad (g) are dfferet. Ca you gve a short explaato as to why ths should be so? OLUTION: I (d), the predcto was for Y ew, a ew observato. Ths ecessarly volved ε ew, a ew error term wth varace. Eve a massve data set wth, say,,, would allow you to avod the mprecso of ε ew. I (g), the task was a cofdece terval for the parameter combato β + β x ew. A large sample sze wll clearly make ths cofdece terval very precse. Ideed, you ca see (g), that lettg wll collapse the terval. () Fd a α cofdece set for the vector the followg facts. β. Ths s ot easy. You ca use β * If W s a m-by- radom ormal vector wth mea ad varace matrx Ω, ad f we assume that Ω s a full-rak m-by-m matrx, the the - dstrbuto of W Ω W s ch-squared wth m degrees of freedom. * If G s dstrbuted as ch-squared o m degrees of freedom, ad f H s dstrbuted as ch-squared o q degrees of freedom, ad f G ad H are G/ m depedet, the the dstrbuto of s F m, q. H / q β Certaly s a radom ormal vector, but t eeds some massage to take β advatage of the frst * fact. OLUTION: ce ˆβ ad ˆβ are lear combatos of ormal radom varables, the Y s, they must have a jot ormal dstrbuto. We kow the meas, varaces, ad the covarace, so we ca assert x x ˆ + β β N, ˆ x β β By subtractg the mea vector ad dvdg by, ths s wrtte as gs

11 x x ˆ + β β N, ˆ x β β Ths matches the frst * ote the problem. Worred about uts? If, say, x s hours ad Y s $, the tercept s βˆ β $ ad the slope s $/hr. The s also $. Thus has ˆ β β o uts o uts uts. The varace matrx above has uts hr. hr hr hr The verse of the varace matrx s eeded, but recall that a b d b b d ad b b a. For the partcular varace matrx, the role of ad b x x s played by +. Thus x x + x x x x + x x + x o uts hr The verse has uts hr hr. Followg the frst * ote (ad usg the symbol G) G ˆ β β x β ˆ β ˆ x ˆ x β β + β β has the dstrbuto Ths s the χ, ch-squared o two degrees of freedom. gs

12 ˆ ˆ ˆ x β β β β β β x x + ˆ G ( ) β β { } ( βˆ β ) + x( βˆ β )( βˆ β ) + ( + x )( βˆ β ) As a fucto of (β, β ), ths s a ellpsod cetered at ( ˆ, ˆ ) β β. The regresso aalyss of varace mea square error, oted as s, has the dstrbuto property ( ) s χ. Ths s depedet of ( βˆ, β ˆ ). We ca use the secod * ote to assert that { ( ) ( )( ) ( )( ) } x x βˆ β + βˆ β βˆ β + + βˆ β ( ) s ( ) ( βˆ β ) + ( βˆ β )( βˆ β ) + ( + )( βˆ β ) x x s s dstrbuted as F,. It follows the that ( βˆ β ) + ( βˆ β )( βˆ β ) + ( + )( βˆ β ) x x α P < F, α s Here F α, s the upper-alpha pot for the F dstrbuto wth (, ) degrees of freedom. The equalty represets the teror of a ellpse, ad ths s detfed as the α cofdece set for (β, β ). There s a oe-to-oe correspodece betwee (β, β ) ad the regresso le. Ths cofdece set ca thus be dsplayed as a cofdece rego for the le. gs